Study on Robustness and Adaptability of Genetic Network Programming with Reinforcement Learning for a Mobile Robot

In the real implementation of autonomous mobile robots, the environments change dynamically with unknown and inexperienced situations, whichmake agents do tasks inappropriately. In order to behave appropriately, the agents should be robust to unknown environments and also have the adaptability mechanisms to change the structures/parameters of their controllers adaptively when inexperienced situations occur. In the classical methods, designing the controllers of the agents using differential equations or conventional network structures have problems, where those methods are difficult to represent all the actual agent behaviors in the dynamic environments. Generally speaking, although the experiences of an expert are needed to design the controllers, the evolutionary algorithms, such as Genetic Algorithm (GA), Genetic Programming (GP) and Genetic Network Programming (GNP) can generate the optimized programs for robot controllers. In order to get the robust controllers, the agents should be trained with a lot of data and the exploration ability should be improved, furthermore, introducing noises may generate various data which improve the ability of the agents to face uncertainty of the environments. On the other hand, the ability of learning the environments using Reinforcement Learning (RL) improves the adaptability to inexperienced troubles in the implementations. The objectives of this thesis are to study the robustness and adaptability of Genetic Network Programming with Reinforcement Learning (GNP-RL), so that the autonomous mobile robot behaves appropriately in the unknown environments and determines its behaviors appropriately to adapt to the troubles caused by broken sensors. Here, GNP is one of evolutionary algorithms to automatic program generation, which generates intelligent rules for the agent behaviors. The structures of GNP are constructed and optimized in the evolution, then the experiences of an expert are not needed. GNP has some advantages compared to the other methods which make GNP suitable for implementing the robots, that is, (1) re-usability of the nodes which make the structures more compact and use small memory; and (2) applicability to Partially Observable Markov Decision Process (POMDP). Furthermore, GNP was enhanced to GNP with Reinforcement Learning (GNP-RL), which has the ability to learn the node transitions and change the functions adaptively by selecting the appropriate sub nodes when troubles occur. Therefore, GNP-RL is applicable for dynamic environments and suitable for robot control. The adaptability of GNP-RL using the Sub node Selection method (SS method) has been analyzed, however, the adaptability mechanism of GNP-RL is not enough when severe troubles occur. In addition, the robustness of GNP-RL in the noisy environments has not been analyzed yet. In order to improve the robustness and adaptability of GNP-RL, this thesis studies 4 topics, that is, (1) Fuzzy GNP with Reinforcement Learning (Fuzzy GNP-RL) which is trained with noises to improve the robust-